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4.3.2 Using the Delta Function 


In this section, we will use the Dirac delta function to analyze mixed random variables. Technically speaking, the Dirac delta function is not 
actually a function. It is what we may call a generalized function. Nevertheless, its definition is intuitive and it simplifies dealing with 
probability distributions. 

Remember that any random variable has a CDF. Thus, we can use the CDF to answer questions regarding discrete, continuous, and mixed 
random variables. On the other hand, the PDF is defined only for continuous random variables, while the PMF is defined only for discrete 
random variables. Using delta functions will allow us to define the PDF for discrete and mixed random variables. Thus, it allows us to unify 
the theory of discrete, continuous, and mixed random variables. 

Dirac Delta Function 


Remember, we cannot define the PDF for a discrete random variable because its CDF has jumps. If we could somehow differentiate the CDF at 
jump points, we would be able to define the PDF for discrete random variables as well. This is the idea behind our effort in this section. Here, 
we will introduce the Dirac delta function and discuss its application to probability distributions. If you are less interested in the derivations, 
you may directly jump to Definition 4.3 and continue from there. Consider the unit step function u(x ) defined by 


r i x>o 

\ 0 otherwise 


( 4 . 8 ) 


This function has a jump at X = 0 . Let us remove the jump and define, for any Ot > 0 , the function U a as 


(*) = 


-) 

9 / 


x>f 


The good thing about U a (x) is that it is a continuous function. Now let us define the function S a (x) as the derivative of U a (x) wherever it 
exists. 


£a(z) 



1*1 < f 
1*1 > f 


Figure 4.10 shows these functions. 
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We notice the following relations: 


8 a (x) 


d r 1 

S"“ w ' 


u(x) = lim u a (x) 

a —>0 


Now, we would like to define the delta "function", $(x) , as 


( 4 . 9 ) 
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8(x) = lim 5 a (x) 

a —>0 


(4.10) 


Note that as OL becomes smaller and smaller, the height of (a?) becomes larger and larger and its width becomes smaller and smaller. Taking 
the limit, we obtain 


8{x) 


oo x = 0 

0 otherwise 


Combining Equations 4.9 and 4.10, we would like to symbolically write 

5(x) = —— u(x). 
dx 

Intuitively, when we are using the delta function, we have in mind S a (x) with extremely small Ol. In particular, we would like to have the 
following definitions. Let g : R i—>• R be a continuous function. We define 


/ oo r p oo 

g(x)8(x — xo)dx = lim / g(x)5 a (x — xo)d, 

•oo a ^° L J- oo 

Then, we have the following lemma, which in fact is the most useful property of the delta function. 

Lemma 4.1 

Let g : R i—> R be a continuous function. We have 


(4.11) 


£ 


g{x)5(x - x 0 )dx = g(x 0 ). 


Proof 


Let I be the value of the above integral. Then, we have 


I = lim a ^ 0 

= lim,. , 0 


f°°oo9( X ) S a(x - X 0 )dx 


r X0+ i M dx 

J x Q -~ a 


By the mean value theorem in calculus, for any OL > 0, we have 

' Xo+ ^ g{x) 


/ 

J Xft 


a 


dx = a^—^~ = g(x a ), 


a 


for some X a € (xq — y> x 0 + y). Thus, we have 

I = lim g(x a ) = g(x 0 ). 

a—>0 

The last equality holds because g(x) is a continuous function and lim Q ._ ) .o x a — #0- 
For example, if we let g{x) = 1 for all X E R, we obtain 


£ 


5{x)dx = 1. 


It is worth noting that the Dirac 8 function is not strictly speaking a valid function. The reason is that there is no function that can satisfy both 
of the conditions 


8{x) = 0( for X ^ 0) 


and 


/ OO 

8{x)dx = 

OO 


1. 


We can think of the delta function as a convenient notation for the integration condition 4.11. The delta function can also be developed 
formally as a generalized function. Now, let us summarize properties of the delta function. 


https://www.probabilitycourse.com/chapter4/4_3_2_delta_function.php 



9/8/2018 


Delta Function I Dirac Delta Function I Generalized PDF 


Definition 4.3: Properties of the delta function 

We define the delta function S(x) as an object with the following properties: 


. f oo x = 0 

1. 5(x) = < 

[ 0 otherwise 

2. 8(x ) = ^ U ( X ), where U ( X ) is the unit step function (Equation 4.8); 

3. f f f 8{x)dx = 1, for any e > 0; 

4. For any e > 0 and any function g{x) that is continuous over ( Xq — €,Xq + e), we have 



g{x)8{x - x 0 )dx = / 

J Xi 


x Q +e 


g(x)8(x 


- x 0 )dx = g(x 0 )- 


Figure 4.11 shows how we represent the delta function. The delta function, 5(x), is shown by an arrow at X = 0. The height of the arrow is 
equal to 1. If we want to represent 2S(x), the height would be equal to 2. In the figure, we also show the function S(x — Xq), which is the 
shifted version of 8(x). 


6(x) 



1 





0 


x 0 


Fig.4.11 - Graphical representation of delta function. 

Using the Delta Function in PDFs of Discrete and Mixed Random Variables 


In this section, we will use the delta function to extend the definition of the PDF to discrete and mixed random variables. Consider a discrete 
random variable X with range Rx — x 3> • • •} an d PMF Px{ x k) ■ Note that the CDF for X can be written as 

F x {x) = ^2 p x{xk)u{x-x k ). 

xk^Rx 

Now that we have symbolically defined the derivative of the step function as the delta function, we can write a PDF for X by "differentiating" 
the CDF: 


fx(x) 


dF x (x) 

dx 


E Xk€Rx p x{x k )£ u {x-x k ) 
E Xke a x p x{x k )8{x-x k ). 


We call this the generalized PDF. 
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For a discrete random variable X with range Rx = .X2, .X3, • • • } and PMF Px{xk ) , we define the (generalized) probability 

density function (PDF) as 

fx{x)= Px{x k )S(x - x k ). 

x k eR x 


Note that for any Xk £ Rx ? the probability of X = Xk is given by the coefficient of the corresponding S function, 8(x — Xk)- 

It is useful to use the generalized PDF because all random variables have a generalized PDF, so we can use the same formulas for discrete, 
continuous, and mixed random variables. If the (generalized) PDF of a random variable can be written as the sum of delta functions, then X is 
a discrete random variable. If the PDF does not include any delta functions, then X is a continuous random variable. Finally, if the PDF has 
both delta functions and non-delta functions, then X is a mixed random variable. Nevertheless, the formulas for probabilities, expectation and 
variance are the same for all kinds of random variables. 

To see how this works, we will consider the calculation of the expected value of a discrete random variable. Remember that the expected value 
of a continuous random variable is given by 


EX = / xfx{x)dx. 

J — OO 

Now suppose that I have a discrete random variable X. We can write 

EX = I°° 0 o X fx( X ) dx 

= f°°oc x p x (x k )S(x - x k )dx 

= E Xk eR x p x(x k ) J'Zo xS(x - X k )dx 

= y^ Xk( Rx x k Px(x k ) by the 4th property in Definition 4.3, 

which is the same as our original definition of expected value for discrete random variables. Let us practice these concepts by looking at an 
example. 


Example 4.16 

Let X be a random variable with the following CDF: 

X > 1 
0 < X < 1 

x < 0 

a. What kind of random variable is X (discrete, continuous, or mixed)? 

b. Find the (generalized) PDF of X. 

c. Find P(X > 0.5), both using the CDF and using the PDF. 

d. Find EX and Var(X). 

• Solution 


Fx(x) = { 


fi + iu-e- 
0 


o a. Let us plot Fx (x) to better understand the problem. Figure 4.12 shows Fx (#). We see that the CDF has two jumps, at 
X = 0 and X = 1. The CDF increases continuously from X = 0 to X = 1 and also after X = 1. Since the CDF is 
neither in the form of a staircase function, nor is it continuous, we conclude that X is a mixed random variable. 
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b. To find the PDF, we need to differentiate the CDF. We must be careful about the points of discontinuity. In particular, we 

have two jumps: one at X = 0 and one at X = 1. The size of the jump for both points is equal to Thus, the CDF has 
two delta functions: j S(x ) + ~ 1). The continuous part of the CDF can be written as -j(l — e~ x ), for X > 0. 

Thus, we conclude 

fx(x) = i 5(x ) + i S(x - 1) + l e -*u(x ). 

c. Using the CDF, we have 

P(X > 0.5) = 1 — Fx(0.5) 

= i+ ip-0.5 
4 2 

= 0.5533 


Using The PDF, we can write 
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P(X>0.5) = f“f x (x)dx 

= Jo” + \ 5 ( x ~ + ^e- x u(x)^dx 

= 0 + j + j f 0 ™ e X dx (using Property 3 in Definition 4.3) 

= \ + i-e-°- 5 = 0.5533 

d. We have 

EX= f°°oo x fx{x)dx 

= fZ (\ x5 ( x ) + \ xS ( x ~ 1 ) + \xe~ x u( x ) S jd x 

= + + J 0 °° xe~ x dx (using Property 4 in Definition 4.3) 


Note that here Xe x dx is just the expected value of an Exponential^ 1) random variable, which we know is 
equal to 1. 

EX 2 = fZ x 2 fx{x)dx 

= fZ ^jX 2 6( x ) + jX 2 ^( x - 1) + \x 2 e~ x u(x)^dx 

= |x0+|xl + | f 0 °° x 2 e~ x dx (using Property 4 in Definition 4.3) 



Again, note that X 2 e X dx is just EX 2 for an Exponential) 1) random variable, which we know is equal to 2. 
Thus, 

Var(X) = EX 2 - {EX) 2 



it 
16 ' 


In general, we can make the following statement: 



«— previous 
next —> 
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